Collimated stereo display system

ABSTRACT

A collimated stereo display system is provided. The system comprises: an image generator enabled to provide a stream of stereoscopic images comprising left eye images and right eye images; at least one image modulator enabled to receive the stream of stereoscopic images from the image generator and form light into the stereoscopic images for viewing at a viewing apparatus for filtering the light into the left eye images and the right eye images respectively thereby providing a three dimensional rendering of the stereoscopic images when viewed through the viewing apparatus; and, collimation apparatus enabled to receive and collimate the light from the at least one image modulator, such that rays of light are substantially parallel when the stereoscopic images are received at the viewing apparatus.

FIELD

The specification relates generally to three dimensional displays, andspecifically to a collimated stereo display system.

BACKGROUND

Three dimensional displays generally suffer from a problem of inducingeye strain in some viewers due to conflicts between accuratepresentation of stereopsis, vergence and accommodation. For example, 3Ddisplays generally require a viewer to always focus at the screen planeto keep it in focus, but depth cues from accommodation are inconsistentwith depth cues from vergence.

For example, as the focal plane remains at the plane of the screen, aviewer's eyes are forced to converge on stereoscopic objects whoseparallax implies a position in space which differs from the screenplane. Consider a screen plane 10 feet away from a viewer. Their eyesremain focused on the screen plane, however a stereoscopic presentationimplies that an object is 15 feet away from the viewer. The object thenmoves from an implied 15 feet away from the viewer to an implied 7 feetaway from the viewer. All the while the eyes remain accommodated/focusedon a fixed screen plane that is 10 feet away.

The human brain has short-cuts in it that allow humans to expect alearned response, or muscle memory level of accommodation/focus. Thesystems of vergence and focus are thus normally cooperative. In a stereosolution with a planar display the viewer is asked to decouple thisrelationship that the viewer has learned to utilize to gauge distance.This decoupling induces fatigue and stress over time.

BRIEF DESCRIPTIONS OF THE DRAWINGS

For a better understanding of the various implementations describedherein and to show more clearly how they may be carried into effect,reference will now be made, by way of example only, to the accompanyingdrawings in which:

FIG. 1 depicts a collimated stereo display system, according tonon-limiting implementations.

FIG. 2 depicts the system of FIG. 1 in use, according to non-limitingimplementations.

FIGS. 3 to 5 depict vergence behaviour of human eyes when focusing onvirtual objects in stereoscopic images projected onto a screen,according to non-limiting implementations.

FIGS. 6 to 8 depict accommodation behaviour of human eyes when focusingon virtual objects in stereoscopic images projected onto a screen,according to non-limiting implementations.

FIGS. 9 to 10 depict collimated stereo display systems, according tonon-limiting implementations.

FIG. 11 depicts a collimated stereo display system with head tracking,according to non-limiting implementations.

FIG. 12 depicts the system of FIG. 11 in use, according to non-limitingimplementations.

FIG. 13 depicts a method of adjusting stereoscopic images in the systemof FIG. 11 when a position of a viewing apparatus changes, according tonon-limiting implementations.

FIG. 14 depicts an in use collimated stereo display system withinterocular distance variation, according to non-limitingimplementations.

FIG. 15 depicts a virtual landscape used in the system of FIG. 14,according to non-limiting implementations.

FIG. 16 depicts changes in stereoscopic images when the interoculardistance changes in the system of FIG. 14, according to non-limitingimplementations.

FIG. 17 depicts insertion of two dimensional images into a stream ofthree dimensional images in the system of FIG. 14, according tonon-limiting implementations.

FIG. 18 depicts accommodation behaviour of human eyes when focusing onvirtual objects in stereoscopic images projected onto a screen,according to non-limiting implementations.

DETAILED DESCRIPTION

An aspect of the specification provides a system comprising: an imagegenerator enabled to provide a stream of stereoscopic images comprisingleft eye images and right eye images; at least one image modulatorenabled to receive the stream of stereoscopic images from the imagegenerator and form light into the stereoscopic images for viewing at aviewing apparatus for filtering the light into the left eye images andthe right eye images respectively thereby providing a three dimensionalrendering of the stereoscopic images when viewed through the viewingapparatus; and, collimation apparatus enabled to receive and collimatethe light from the at least one image modulator, such that rays of lightare substantially parallel when the stereoscopic images are received atthe viewing apparatus.

The system can further comprise: at least one projector for projectingthe stereoscopic images, the at least one projector comprising the atleast one image modulator; a least one screen upon which thestereoscopic images are projected; and at least one collimating mirrorfor reflecting the stereoscopic images from the at least one screen suchthat reflected light is collimated, the collimating apparatus comprisingthe at least one screen and the at least one collimating mirror. The atleast one collimating mirror can be one of spherical and parabolic, andthe at least one screen can be a shape complimentary to the at least onecollimating mirror.

The collimating apparatus can comprise: a least one screen upon whichthe stereoscopic images are provided; and at least one collimatingmirror for reflecting the stereoscopic images from the at least onescreen such that reflected light is collimated.

The system can further comprise a computing device enabled for at leastone of: warping the stereoscopic images prior to the at least one imagemodulator forming the light into the stereoscopic images; repeating thewarping based on a changed position of the viewing apparatus relative tothe collimation apparatus; varying an interocular distance between theleft eye images and the right eye images; and inserting two dimensionalimages into the stream of stereoscopic images.

The system can further comprise an immersive environment, such that thestereoscopic images are associated with an immersive experience in theimmersive environment. The immersive environment can comprise one ormore of a simulation environment, and a flight simulator, and a drivingsimulator. The immersive environment can comprise one or more of avisualization environment, a microscopy visualization environment, amedical imaging visualization environment, an oil and gas visualizationenvironment, and a seismology visualization environment. The immersiveenvironment can comprise a training environment.

The system can further comprise: head tracking apparatus enabled toacquire a position of the viewing apparatus; and a processor enabled toadjust the stereoscopic images for viewing based on one or more of theposition and the orientation of the viewing apparatus. The stereoscopicimages can be projected onto a curved surface for viewing and theprocessor can be further enabled to adjust the stereoscopic images basedon a geometry of the curved surface. The system can further comprise amemory for storing data indicative of the geometry. The processor can befurther enabled to adjust the stereoscopic images based on the geometryof the curved surface by: determining images for display, the imagescomprising the stereoscopic images when they are to be provided on aflat screen; determining a distortion of the flat screen based on thedata indicative of the geometry of the curved surface and the positionof the viewing apparatus; and adjusting the images based on thedistortion such that the stereoscopic images will appear undistortedwhen projected onto the curved surface. Adjusting the stereoscopicimages can be repeated each time a change in position of the viewingapparatus is determined. The collimating apparatus can comprise acollimating mirror and the collimating mirror can be enabled to reflectthe light from the curved surface such that reflected light iscollimated.

The system can further comprise: a processor enabled to adjust thestereoscopic images by varying interocular distance of the left eyeimages and the right eye images. The interocular distance can be variedbased on a virtual distance between a viewer of the stereoscopic imagesand at least one given feature in the stereoscopic images. Theinterocular distance can be varied such that the left eye images and theright eye images are provided in hyperstereo. The left eye images andthe right eye images can be provided in hyperstereo when the virtualdistance is one or more of above a first given threshold value and belowa second given threshold value.

The system can further comprise: a processor enabled to insert twodimensional images into the stream of stereoscopic images based on avirtual distance between a viewer of the stereoscopic images and atleast one given feature in the stereoscopic images. The processor can befurther enabled to insert the two dimensional images into the stream ofstereoscopic images by replacing one of the left eye images and theright eye images with the other of the right eye images and the left eyeimages. The processor can be further enabled to insert the twodimensional images into the stream of stereoscopic images when thevirtual distance is one or more of above a first given threshold valueand below a second given threshold value.

Attention is directed to FIG. 1, which depicts a collimated stereodisplay system 100, according to non-limiting implementations, and FIG.2, depicts system 100 in use. System 100 generally comprises an imagegenerator 101 in communication with at least one image modulator 103.Image generator 101 is generally enabled to provide a stream ofstereoscopic images 105 comprising left eye images 106 a and right eyeimages 106 b. It is also appreciated that while left eye images 106 aand right eye images 106 b are labelled only once in FIG. 1, andsubsequent figures, they are nonetheless present in stereoscopic images105.

As depicted, image generator 101 is in communication with at least oneimage modulator 103 via an optional computing device for processingstereoscopic images 105, as will be presently explained. Computingdevice 104 can comprise any suitable computing device, including but notlimited to a warping engine, a video processing device, a personalcomputer (PC), a server, and the like. In general, computing device 104comprises a processor 112 and a memory 114 (which can comprise volatileand non-volatile storage), as well as any suitable communicationinterfaces, input devices and display devices.

Image generator 101 can comprise any suitable image generator, includingbut not limited to, a storage device for storing three dimensionalcinema files, three dimensional video files, three dimensional displayfiles, a video game systems, a simulation system, a visualizationsystem, a training system, and the like.

At least one image modulator 103 is generally enabled to receive thestream of stereoscopic images 105 from image generator 101 and formlight 107 into stereoscopic images 105 for viewing at a viewingapparatus 109. For example, in depicted implementations, at least oneimage modulator 103 comprises a projector for projecting stereoscopicimages 105. Hereafter at least one image modulator 103 will also bereferred to as image modulator 103 without undue limitation as it isappreciated that system 100 can comprise one or more image modulators103. Furthermore, this convention will be used throughout thespecification.

Image modulator 103, image generator 101 and computing device 104, ifpresent, can be in wired or wireless communication, as desired, usingany suitable protocol and can be local or remote from one another.

Viewing apparatus 109 is generally enabled for filtering light 107 intoleft eye images 106 a and right eye images 106 b, respectively therebyproviding a three dimensional rendering of stereoscopic images 105 whenviewed through viewing apparatus 109.

In other words, system generally comprises a “3D” (“three-dimensional”)system for viewing images in three dimensions, and hence image modulator103 comprises a “3D” projector for projecting stereoscopic images 105that can be separated into left eye images 106 a and right eye images106 b by viewing apparatus 109 (“3D” glasses), using any suitabletechnology. Indeed, it is generally appreciated that viewing apparatus109 is compatible with image modulator 105. For example, inimplementations where image modulator 103 projects left eye images 106 aand right eye images 106 b with different polarization states (includingbut not limited to orthogonal linear polarization states andclockwise/counter-clockwise circular polarization states), viewingapparatus 109 comprises suitable polarized filters for filtering lefteye images 106 a and right eye images 106 b into the respective eye of aviewer 111. Similarly, in implementations where image modulator 103alternately projects left eye images 106 a and right eye images 106 b,one after the other, viewing apparatus 109 comprises suitable electronicshutters (including but not limited to LCD (liquid crystal display)based shutters) which are opened and closed accordingly as each of lefteye images 106 a and right eye images 106 b are projected, thereby forfiltering left eye images 106 a and right eye images 106 b into therespective eye of a viewer 111. In any event, it is appreciated that themethod of providing stereoscopic images 105 for viewing via viewingapparatus 109 is not to be considered particularly limiting. Forexample, other implementations include, but are not limited to active orpassive stereo display systems using color comb filters and/or polarizerfilters.

It is further appreciated that, while viewing apparatus 109 are depictedin FIG. 1 as facing out of the page, such a depiction is schematic onlyand viewer 111 normally, substantially faces mirror 117 to view images105.

It is generally appreciated that stereoscopic images 105 are to beprovided at a suitable surface for viewing, such as a screen. However,in contrast to prior art 3D systems, system 100 further comprises acollimation apparatus 113 enabled to receive and collimate light 107from image modulator 103, such that rays 121 of light 107 aresubstantially parallel when stereoscopic images 105 are received atviewing apparatus 109. In depicted non-limiting implementations,collimation apparatus 113 comprises: at least one screen 115 upon whichstereoscopic images 105 are projected; and at least one collimatingmirror 117 for reflecting stereoscopic images 105 from screen 115 suchthat reflected light 119 is collimated.

Screen 115 can comprise any suitable screen enabled to scatter/reflectlight 107 which is then subsequently reflected by collimating mirror117. It is appreciated that screen 115 is generally convex, including,but not limited to, generally spherical and generally parabolic. Use ofa convex screen provides advantages over prior art two-dimensionalsimulators, which tend to used concave screens upon which images areviewed. For example, concave screens suffer from cross talk from side toside and top to bottom as light from images on each side are reflectedoff of an opposite side of the concave screen. This leads to degradedcontrast, flare and ghosting. It is furthermore appreciated that suchproblems can be made worse via use of back projection and/or rearprojection onto concave screens. For example, rear projection concavescreens usually used in flight simulators are widely misunderstood. Theyare often falsely cited as being brighter and with more contrast thanfront projection systems, but that is really only in high ambient light.On the contrary, many rear projection screens suffer from severeghosting off axis rays, due to geometric scattering and screenabsorption, causing double images and flare. In contrast, by using aconvex front projection screen, the cross talk problem is eliminated,which leads to improved contrast, improved brightness, improvedresolution, lower cost, relaxed projector placement, reduced flare,reduced ghosting and the like.

Collimating mirror 117 can comprise any suitable concave mirror,including but not limited to a spherical mirror and a parabolic mirror,which has the general property of reflecting light in substantiallyparallel rays 121. Hence, in contrast to prior art 3D systems,light/parallel rays 121 forming stereoscopic images 105 is collimatedwhen arriving at viewing apparatus 109 such that eyes of viewer 111 arenot focussed at screen 115 or collimating mirror 117 but at infinity.The advantages of this are described below with reference to FIGS. 3 to8.

However, it is appreciated that parabolic mirrors can provide bettercollimation than a spherical mirror. However parabolic mirrors can beexpensive to manufacture while spherical mirrors are cheaper and easierto manufacture though spherical mirrors can cause some coma at edges.Nonetheless, spherical mirrors can be an adequate trade-off betweendegree of collimation and expense. Hence, it is appreciated thatcollimation of collimating mirror 117 does not need to be perfect.

Further, collimating mirror 117 can comprise any suitable material,including but not limited to glass, and Mylar™ film. In the later case,it can be challenging to manufacture Mylar™ film mirrors which areactually perfectly parabolic or perfectly spherical, and in generalMylar™ film mirrors generally comprise a boundary and pressure limitedcatenary curve often with some saddle bulging in the middle, for crosscockpit wide field of view displays. However, while performance andcomfort decrease as more aberrations appear with lesser quality mirrors,the degree of collimation is generally adequate.

It is furthermore appreciated that screen 115 has a shape complimentaryto collimating mirror 117, and hence screen 115 is generally curved. Forexample, as collimating mirror 117 is generally concave, screen 115 isgenerally convex: e.g. in implementations where collimating mirror 117is generally spherical and concave, screen 115 is also generallyspherical and convex, and in implementations where collimating mirror117 is generally parabolic and concave, screen 115 is generallyparabolic and convex. However, it is appreciated that screen 115 neednot be exactly complimentary to collimating mirror 117. For example, inimplementations where collimating mirror 117 is spherical, screen 115could be parabolic and vice versa, with corrections to images 105performed by computing device 104 to correct for the resultingaberrations in the viewed images.

Indeed, it is further appreciated that deviations from ideal concavemirrors a convex screens are within the scope of presentimplementations. While such a system may not be ideal, such a systemcould still provide advantages with collimated stereoscopic displaysdescribed herein. For example, in some implementations, images 105reflected from a flat screen (i.e. screen 115) could be viewed using awide radius parabolic mirror (i.e. collimating mirror 117). In some ofthese implementations, corrections to images 105 can be performed bycomputing device 104 to correct for the resulting aberrations in theviewed images.

It is yet further appreciated that collimating mirror 117 is arrangedsuch that parallel rays 121 that arrive at a design eye point (DEP) 123that is generally aligned with eyes of viewer 111. It is furthermoreappreciated that tracing parallel rays 121 back from DEP 123, yieldsconvergence on points at screen 115, in focus for viewer 111. In otherwords, as depicted, each parallel ray 121 can be traced back to screen115 via respective reflected light 119.

Further, it is appreciated that, in some implementations, a bottom ofscreen 115 can be clear or close to clear from a top portion ofcollimating mirror 117. In other words, screen 115 and collimatingmirror 117 are arranged such that stereoscopic images 105 projected ontoscreen 115 are viewable by viewer 111, such that a top portion ofcollimating mirror 117 is not blocked by screen 115.

In depicted implementations, system 100 comprises computing device 104enabled for warping stereoscopic images 105 prior to image modulator 103forming light 107 into stereoscopic images 105. For example,stereoscopic images 105 are warped at computing device 104 such thatwhen stereoscopic images 105 are projected onto curved screen 115 andviewed at viewing apparatus 109, via parallel rays 121, stereoscopicimages 105 do not appear curved.

In further implementations, as will presently be described, computingdevice 104 can be further enabled for repeating the warping based on achanged position of viewing apparatus 109 relative to collimationapparatus 113. In other words, system 100 can further comprise headtracking apparatus, described below with reference to FIGS. 11 and 12,which tracks a position of viewing apparatus 109 and/or a head of viewer111, and dynamically warps stereoscopic images 105 based on the positionof viewing apparatus 109 and/or a head of viewer 111, to provide achanged viewpoint.

In yet further implementations, as will presently be described,computing device 104 can be further enabled for at least one of: varyingan interocular distance (which can also be referred to as interpupilarydistance) between left eye images 106 a and right eye images 106 b; and,inserting two dimensional images into stream of stereoscopic images 105.

However, in other implementations, image generator 101 can be enabled toperform all the functions of computing device 104; hence, in theseimplementations, image generator 101 and computing device 104 arecombined into one device and image generator 101 then comprisescomputing device 104.

In some implementations, system 100 can be used for general 3D videoapplications, including but not limited to, 3D collimated cinemaapplications, collimated 3D displays, one person desktop collimated 3Ddisplays, 3D collimated television, and 3D collimated video gamesystems.

It is further appreciated that system 100 can further comprise animmersive environment, such that stereoscopic images 105 are associatedwith an immersive experience in the immersive environment.

For example, the immersive environment comprises a simulationenvironment, including but not limited to a flight simulator, a drivingsimulator, a spaceship simulator, and the like. Hence, system 100 cancomprise “real world” controls, as in a flight simulator, such thatviewer 111 can control stereoscopic images 105 to provide, for example,a simulation of flight. Stereoscopic images 105 can hence be dynamicallyupdated, in a feedback loop with image generator 101 (and optionallycomputing device 104), to comprise images of a “real world” simulationto replicate the real world, such that viewer 111 is interacting withthe simulation environment in manner similar to interacting with thereal world.

It has heretofore been conventionally understood that three dimensionaleffects were of no use in collimated display systems, such as simulationenvironments including flight simulators, as collimated light is assumedto arrive from an “optical infinity”, which is generally taken as about9 meters (about 30 feet). For example optical infinity is generallyassumed to be a distance (for an average adult person) at which theangle of view of an object at that distance is effectively the same fromboth the left and right eyes, and hence it is further assumed that thereis little advantage in using two-channel imagery and stereoscopicdisplay systems. However, testing of a working prototype of system 100configured as a flight simulator has shown these assumptions of theprior art to be false. Indeed, pilots interacting with the workingprototype found the interactive experience provided by a collimatedstereo display system to be more realistic than a collimated non-stereodisplay system.

Firstly, even at distances ranging from about 4 feet to at least about400 meters (e.g. about ¼ of a mile) simulated features were produciblewith different views for the left and right eyes, that provided anoticeable and improved three dimensional effect with better depthperception.

Secondly, better resolution of the flight simulator was perceived whenusing stereoscopic images rather than two dimensional images, largelydue to super-resolution effects; in other words, the human brain is ableto superimpose the stereoscopic left and right eye images, andinterpolate a higher resolution for the resulting image than if a singleimage was used for both the left and right eyes (i.e. two dimensional).

Thirdly, there are features that can be provided in flight simulatorsthat are around 9 meters from a design eye point including, but notlimited to, mid-air refuelling booms, low flying helicopters, drivingsimulators and landscape features when simulating landing. However, suchfeatures can be provided at greater than 9 meters and less than meters.In these situations pilots noted that three dimensional stereoscopicimages provided a much more realistic experience than two dimensionalimagery.

In further implementations, the immersive environment comprises avisualization environment, which is not necessarily related to a realworld environment. For example, the visualization environment caninclude, but is not limited to, one or more of a CAVE (Cave AutomaticVirtual Environment), a medical imaging visualization environment, anoil and gas visualization environment, a seismology visualizationenvironment and the like. In these implementations, system 100 cancomprise controls to “manipulate” the visualization environment. Forexample such a visualization environment, and associated controls, canbe used to visualize and manipulate virtual medical molecules/drugsetc., oil deposits, seismic plates and the like, in stereoscopic images105. It is further appreciated that collimated display systems have notheretofore been used in visualization environments, and certainly notcombined with stereoscopic images. Indeed, within the general field ofdisplay systems, there tends to be little or no overlap betweensimulation environments and visualization environments.

In yet further implementations, the immersive environment comprises atraining environment, including but not limited to training environmentsfor repairing equipment and/or building equipment, and the like (e.g.industrial vehicle building and repair, as non-limiting example). Forexample, such a training environment, and associated controls, can beused to visualize and manipulate virtual items in stereoscopic images105 a for repairing and/or building equipment. It is further appreciatedthat collimated display systems have not heretofore been used intraining environments, and certainly not combined with stereoscopicimages.

General advantages of the collimated stereo display are now discussedwith reference to FIGS. 3 to 5 and FIGS. 6 to 8, and 18. Specifically,FIGS. 3 and 4 depict vergence behaviour of human eyes when focusing onvirtual objects in stereoscopic images projected onto a screen, when thevirtual objects are respectively “located” ahead of the screen andbehind the screen. From FIG. 3 it is appreciated that the left eye andthe right eye attempt to cross when the object P3 appears ahead of thescreen (i.e. vergence), but at the same time the left eye image P1 andright eye image P2 are actually projected on the screen, which areconflicting stereoscopic cues which can cause headaches and nausea in aviewer. It is appreciated that the degree of discomfort can vary fromviewer to viewer. With reference to FIG. 4, the problem becomes lessacute when the object P3 is “located” behind the screen, as the eyesstill attempt to converge. However, with reference to FIG. 5, and as inpresent implementations, when the object P3 (not depicted in FIG. 5) isnominally located at optical infinity, the vergence cues are generallyin agreement with the actual location of the images P1, P2 (i.e. at thescreen) and the eye axes are generally parallel, thereby reducing strainon the eye. Indeed, such a view generally simulates a real view out areal window—i.e. views separated by interocular (e.g. interpupilary)distance and comfortably relaxed, with axes substantially parallel,thereby reducing strain on the eye.

Further, focus of the eye (accommodation) is also in conflict whenvirtual objects in stereoscopic images are respectively “located” aheadof the screen and behind the screen For example, FIGS. 6 and 7 depictaccommodation behaviour of human eyes when focusing on virtual objectsin stereoscopic images projected onto a screen, when the virtual objectsare respectively “located” ahead of the screen and behind the screen.With reference to FIG. 6, when the object is virtually located ahead ofthe screen, the eye attempts to focus ahead of the screen such that thefront surface of the eye tenses, as do the ciliary muscles to force theeye lens to the curvature necessary to focus the image on the retina.

With reference to FIG. 7, the problem becomes slightly less acute whenthe object is “located” behind the screen, as the eyes and ciliarymuscles undergo modest tension. However, the conflict betweenaccommodation and vergence is still present.

However, with reference to FIG. 8, and as in present implementations,when the object (not depicted in FIG. 8) is nominally located at opticalinfinity, the accommodation cues are generally in agreement with theactual location of the left and right eye images (i.e. at the screen)and the eye and ciliary muscles are generally relaxed, thereby reducingstrain on the eye.

However, as discussed above, even when many objects are nominallylocated at optical infinity, by providing a collimated stereo displaysystem 100, there are noticeable differences in view for the left andright eye images, and good stereoscopic images can be formed in system100, hence providing a more comfortable three dimensional viewingexperience over long periods of time without fatigue.

In other words, FIGS. 6, 7, and 8 are three “normal cases” of lookingout a window. To further illustrate the issues addressed by the presentspecification, attention is directed to FIG. 18, which depicts an eyefocusing on a window or screen plane (which is the normal case for allexisting 3D displays). Indeed, while the human eye can focus on closeobjects, with 3D displays depth cues are in conflict, especially whenobjects are “virtually” depicted in front a screen when the imagethereof is actually presented on the screen.

For example, when a viewer is watching 3D images e.g. games on acomputer screen about 18 inches in front of the viewer, the viewer'sfocus is fixed, with the front lens surface being fairly steep andciliary muscles contracted as in FIG. 6. But the stereo game, orsimulation, or microscope or whatever is being displayed in 3D can bedepicting an object (e.g. an aircraft) at infinity with proper vergencecues such as those depicted in FIG. 5. So the stronger vergence cue ofstereo disparity on retinas says the image is at infinity, but theweaker depth cue from accommodation says it is actually only 18 inchesaway. It is this conflict in depth cues which causes eye strain.

It is even more eye straining and “unnatural” if the vergence cue aresimilar to those of FIG. 3, for an object in front of the computerscreen, for example about 12 inches away when the focus accommodationcue are telling the viewer's brain that the object is still about 18inches away.

Such a conflict is somewhat resolved by present implementations, asgreater comfort and a more natural view of distant objects is achievedwhen both accommodation and vergence cues are generally in sync and theobjects are “pushed” further away by collimating mirror 117 such thatinfinity looks like a real infinity out a window—with eyes relaxed bothin vergence (uncrossed or parallel) and focus (i.e. relaxed flatterlenses). This makes it easier for a viewer to watch a simulated view forhours at a time with less eyestrain, even for viewers who normally havea lot of difficulty watching 3D images.

Attention is next directed to FIG. 9, which depicts a system 100 asubstantially similar to system 100, with like elements having likenumbers, however with an “a” appended thereto. For example imagemodulator 100 a is similar to image modular 100. However, in theseimplementations, screen 115 a comprises a rear projection screen(sometimes referred to as a back projection screen (BPS)) and imagemodulator 100 a is arranged to project onto a rear of screen 115 a, withrespective stereoscopic images being reflected by collimating mirror 117a. In other words, system 100 a is similar to system 100 with, however,a rear projection geometry rather than a front projection geometry.

Attention is next directed to FIG. 10, which depicts a system 100 bsubstantially similar to system 100, with like elements having likenumbers, however with a “b” appended thereto. For example collimatingmirror 117 b is similar to collimating mirror 117. However, in theseimplementations, image modulator 103 b is combined with a screen: inother words, image modulator 103 b comprises a curved display (forexample, based on light emitting diodes (LEDs), organic LEDs or anyother suitable technology for producing curved displays), withrespective stereoscopic images formed there upon, which are in turnreflected by collimating mirror 117 b. In other words, system 100 b issimilar to system 100 with, however, the projection elements replaced bycurved panel display technology. Indeed, it is appreciated that use ofcurved panel display technology can enable smaller near-the-eye displaysof lower power and size than large simulator BPS screens used today inflight simulators.

Attention is next directed to FIG. 11, which depicts a system 100 csubstantially similar to system 100, with like elements having likenumbers, however with a “c” appended thereto. For example imagemodulator 100 c is similar to image modular 100. FIG. 12 depicts system100 in use. However, system 100 c further comprises head trackingapparatus 1101, in communication with computing device 104 c, headtracking apparatus enabled to acquire a position of viewing apparatus109 c including but not limited to a position of a head of viewer 111 c.It is appreciated that a position of viewing apparatus 109 c caninclude, but is not limited to, a position of viewing apparatus 109 c inthe area in front of collimating mirror 117 c and an orientation ofviewing apparatus 109 c (e.g., whether viewer 111 c is looking left,right, up, down, straight ahead and the like). FIGS. 11 and 12 depictfurther details of computing device 104 c, described in further detailbelow. However, it is appreciated that computing device 104 c comprisesa processor 1103 and a memory 1105, similar to processor 112 and memory114 respectively.

In depicted in non-limiting example implementations, head trackingapparatus 1101 comprises one or more—of a digital camera and a digitalvideo camera (generically a camera) oriented to take pictures of viewingapparatus 109 c and/or a head of viewer 111 c (e.g. an area in front ofcollimating mirror 117 c). However, it is appreciated that any suitablehead tracking apparatus is within the scope of present implementations,and the specification is not to be unduly limited to a camera. In someimplementations, head tracking apparatus can further comprise acomputing device (not depicted) for processing head tracking data todetermine a position of viewing apparatus 109 c; data indicative of theposition of viewing apparatus 109 c can then be communicated tocomputing device 104 c.

However, in other implementations, once head tracking apparatus 1101acquires a position of viewing apparatus 109 c, the acquired data iscommunicated to computing device 104 c for processing, such that aprocessor 1103 determines a position and of viewing apparatus 109 c byprocessing the acquired data.

Indeed, it is appreciated that processor 1103, which is generallyenabled to adjust stereoscopic images 105 c for viewing based on theposition of viewing apparatus 109 c. For example, when the position ofviewing apparatus 109 c changes, stereoscopic images 105 c are adjustedso that viewer 111 c is presented with a view commensurate with thechange in position. For example, when viewer 111 c steps left or right,and/or moves his/her head up or down, the parallax and view instereoscopic images 105 c can be adjusted to reflect the change inposition and/or orientation.

It is further appreciated, however, that as stereoscopic images 105 care projected onto a curved surface (i.e. screen 115 c) for viewing andprocessor 1103 is hence further enabled to adjust stereoscopic images105 c based on a geometry of the curved surface. In other words,processor 1103 is generally enabled to both change the view representedby stereoscopic images 105 c and adjust for the curved surface in thechanged view.

It is further appreciated that head tracking apparatus 1101 can one ormore of communicate with computing device 104 c when a change inposition of viewing apparatus 109 c is detected and periodicallycommunicate a position of viewing apparatus 109 c to computing device104 c, whether the position has changed or not, such that changes inposition can be determined by differences in position that occur fromcommunication to communication.

Attention is now directed to FIG. 13 which depicts a method 1300 ofadjusting stereoscopic images 105 c when a position of viewing apparatus109 c changes. In order to assist in the explanation of method 1300, itwill be assumed that method 1300 is performed using system 100 c.Furthermore, the following discussion of method 1300 will lead to afurther understanding of system 100 c and its various components.However, it is to be understood that system 100 c and/or method 1300 canbe varied, and need not work exactly as discussed herein in conjunctionwith each other, and that such variations are within the scope ofpresent implementations.

Method 1300 further assumes that data 1201 indicative of a geometry ofsystem 100 has been stored at memory 1105, data 1201 including but notlimited to data indicative of the geometry of the curved surface (i.e.screen 115 c), including but not limited to a curvature of screen 115 c,and relative position(s) of image modulator 103 c, screen 115 c, andcollimating mirror 117 c, as well as a position of nominal DEP 111 c,which is assumed to change when the position and/or orientation ofviewing apparatus 109 c changes.

At block 1301, processor 1103 receives data indicative that the positionof viewing apparatus 109 c has changed. For example, head trackingapparatus 1101 detects a change in position of viewing apparatus 109 cand transmits data indicative of such to processor 1103.

At block 1303, processor 1103 determines images for display, the imagescomprising stereoscopic images 105 c when they are to be projected ontoa flat screen based on the changed position. For example, when viewingapparatus 109 c moves up, down, left, right, etc., DEP 123 c changes,which can be reflected in changes in viewing angle of virtual featuresin stereoscopic images 105 c, as well as changes in parallax and thelike. Processor 1103 is hence enabled to process stereoscopic images 105c to reflect these changes, as if stereoscopic images 105 c were to beprovided on a flat surface (i.e. not curved).

At block 1305, processor 1103 determines a distortion of the flatsurface based on the data indicative of the geometry of the curvedsurface (i.e. screen 115 c) and the position of viewing apparatus 109 c.In other words, the flat surface is processed as a distortable mesh, anddistorted to reflect the shape of screen 115 c, when stereoscopic images105 c are viewed from the changed position viewing apparatus 109 c.

At block 1307, the images produced at block 1301 are adjusted based onthe distortion at block 1305, such that stereoscopic images 105 c willappear undistorted when projected onto the curved surface.

In contrast, conventional display generators produce content that isrendered for a single point of perspective. This content is usuallygenerated under the assumption that the display plane is a planarsurface and that a simple view frustum can describe the display fieldsof view, with viewers in fixed positions in space simplifying thedrawing of geometry in a simulated scene.

It is appreciated that to achieve a wide field of view, collimateddisplays generally employ a curved projection surface. Image warpingallows the usually planar target to be treated as if it was a flexiblemesh, allowing a uniform and appropriate distortion of the image so thatonce relayed off the curved screen to the curved optics in thecollimated display solution, such as a collimating mirror, the resultwill be an image free of bowing or lensing distortions from the viewer'spoint of perspective.

However, such systems do not take into account that the viewer'sposition can and will vary. To address this, the present specificationprovides a solution of accommodating movement through a combination oftracking a viewer's position in space and correcting the imagedistortion on-the-fly, as in method 1300.

It is appreciated that stereopsis uses, for its three dimensional depthcures: the fusing of two unique perspective views (i.e. left eye andright eye); the monocular depth cue of accommodation (focus); and thebinocular depth cue of convergence. With motion parallax added (throughthe point of perspective tracking described herein), all of themonocular depth cues humans have available are then available toreinforce stereoscopic depth cues.

It is hence appreciated that monocular depth cues that can be tetheredto a stereoscopic experience include:

Perspective (a point of view from the viewer's position in space.Present implementations enable perspective to be altered as in the realworld as the viewer moves).

Depth from motion (motion parallax and alteration in object size in thevisible scene, which is generally tied to perspective. For example, whenviewing trees in a forest, trees are positioned at differing planesrelative to the viewer's position; as the viewer moves, close treesappear to move quickly within the viewer's field of view, and treesfarther away move more slowly)

Occlusion (The effect moving about changes the visibility of objectsthat are occluded by one another. For example, trees in a forest arepositioned at differing planes relative to the viewer's position; as theviewer moves, trees that are farther away can be occluded by treescloser to the viewer).

Hence, by adjusting images 105 based on a position of viewer 111 c, allof the afore mentioned depth cues can be integrated into system 100 c,which enables the virtual experience to appear much more lifelike

It is appreciated that method 1300 can be repeated each time a change inposition of viewing apparatus 109 c is determined. Hence, stereoscopicimages 105 c can be dynamically adjusted as viewer 111 c moves relativeto collimating mirror 117 c, providing an overall enhanced 3Denvironment.

Indeed, it is appreciated that the addition of motion parallax cues canbe very powerful additional depth cue, extending far beyondaccommodation, which is generally limited to one minute of arc by thedepth of focus calculated from a 2 mm pupil in good light. Motionparallax from head movement can provide a stronger cue than eyeseparation and vergence since the “baseline” for images with stereodisparity can be longer. For example if a viewer moves their head 2 or 3times to view a given scene, the greater depth and vernier acuity can beresolved over the usual 64 mm nominal eye spacing due to the motionparallax. For example, even viewers with one bad eye can perceive depthby head motion and accommodation if they have good head tracking Motionparallax can then takes the place of vergence cues.

It is further appreciated that, in some implementations, head trackingcan be turned on and off. For example, for multiple viewer systems (suchas cross cockpit collimated simulators with pilot and co-pilot in anon-limiting example), head tracking may not be desirable unless only asingle viewer is using the system. Hence, in these implementations, headtracking can be turned on for a single viewer and turned off formultiple viewers.

Attention is next directed to FIG. 14, which depicts an “in-use” system100 c substantially similar to system 100 c, with like elements havinglike numbers, however with a “d” appended thereto, rather than a “c”.For example computing device 104 d is similar to computing device 104 c.However, in these implementations, a head tracking device is optional(and indeed not depicted), and processor 1103 d is enabled to adjuststereoscopic images 105 d by varying interocular distance between lefteye images 106 a d and right eye images 106 bd. For example, as depictedin profile in FIG. 15, a “virtual” landscape 1500 can include a virtualfeature 1501, for example a geographic feature such as a hill or amountain. DEP 123 d is depicted as in the “sky” in FIG. 15 as it assumedthat landscape 1500 is part of a flight simulation system and henceviewer 111 d would be located above landscape 1500. A virtual distance1503 can be determined between DEP 123 d (i.e. viewer 111 d) and feature1501.

It is appreciated that landscape 1500 would generally not be “seen” asdepicted in FIG. 15 by viewer 111 d, but rather stereoscopic images 105d would be generated to represent a view of landscape 1500 as seen fromDEP 123 d. Generally, stereoscopic images 105 d are generated based on ahuman norm interocular distance of about 64 mm (though the human norminterocular distance is generally appreciated to be in a range of about62 mm to about 65 mm hence any suitable interocular distance can bechosen as representing the norm; however human interocular distances canrange from about 58 mm to about 71 mm). However, as depicted in FIG. 16,the interocular distance between left eye images 106 ad and right eyeimages 106 bd can be varied, for example increased or decreased,depending on virtual distance 1503, to produce updated left eye images106 ad′ and right eye images 106 bd′ based on a new interoculardistance. For example, as depicted in FIG. 16, the interocular distancecan be increased such that left eye images 106 ad′ and right eye images106 bd′ are provided in hyperstereo, which is generally appreciated toprovide more optical depth information to a viewer than standard stereoimages. In other words, in these implementations, system 100 d isenabled to enter a hyperstereo mode.

It is appreciated that hyperstereo can affects a sense of scale, but isuseful for increasing the stereo disparity of distant objects. Forexample, hyperstereo is used in aerial photographs with very long baselines to produce topographical maps. It is also useful with binocularsor rangefinders to effectively increase the interocular distance andbaseline to judge relative depth. Furthermore, at distances of greaterthan about 100 m, interocular distance of several hundred feet do notgenerally affect distance perception considerably, but increases thestereo disparity of distant objects. Hence, providing hyperstereo imagesof landscape, for example, can provide enhanced details of virtualfeatures. Indeed hyperstereo baseline images of earth orbits (e.g. fromone side of planet orbit to the other) has been used for scientific 3Dstudies of the moon.

In some implementations, processor 1103 d can be enabled to provide lefteye images 106 a d and right eye images 106 bd in hyperstereo whenvirtual distance 1503 is one or more of above a first given thresholdvalue and below a second given threshold value. For example, a firstgiven threshold value can be around 100 meters, and a second giventhreshold value can be around 1000 meters, however any suitablethresholds are within the scope of present implementations.

For example, baselines below 64 mm can be useful for both micro andmacro work, such as examining bugs or blood cells. However, whenvisualizations of, for example, galaxies are provided, baselines oflight years and/or parsecs and/or astronomical units can be within thescope of present implementations. Hence, it is appreciated that thereare no particular limitations on interocular distance in presentimplementations, which can hence range from zero (for two-dimensionalimages, as described below, to extremely large distances). It is furtherappreciated that eyepoints grow equally positive and negative to the“normal” interocular distance of about −32 mm for the left eye and +32mm for the right eye (e.g. a total of about 64 mm). However, the secondthreshold value can also be less than the first threshold value suchthat hyperstereo is provided at small distances and at large distances:for example, the second threshold value can be less than 9 meters suchthat virtual features close to DEP 123 d appear larger than life toprovide more optical detail than would normally be available when theinterocular distance was at human norm.

It is further appreciated that while examples of varying interoculardistance heretofore have all involved flight simulators, varyinginterocular distance can be used in any suitable immersive environment,as described above. For example, a variable interocular distance caneither be manually dialled in or automatically change depending on theactual distances being viewed in a simulated scene (e.g. when a vieweris closer to the ground, a more natural interocular distance can beused, and when a viewer is at higher altitudes a hyperstereo setting canbe used).

In some of these implementations, an interocular distance of less thanhuman norm can be used, as also depicted in FIG. 16 (see left eye images106 ad″ and right eye images 106 bd″). For example when imaging drugs ina medical visualization environment, or any other very small features inany other suitable immersive environment, where human norm would notyield an adequate view of the feature, an interocular distance of lessthan human norm can be used. In other words, system 100 d can also beenabled to enter a hypostereo mode which can be useful for very closework viewing such as with or microscopy or macro applications, or thelike.

In general, it is appreciated that interocular distance can be varied inany suitable manner, including but not limited to based on thresholdsand/or virtual distances, for any suitable immersive environment.

It is yet further appreciated that as human interocular distances canvary from person to person, interocular distance for system 100 d can bealso be varied based on an interocular distance of a given viewer. Forexample, interocular distance for system 100 d can be fixed for givenviewer, but then changed for a different viewer to accommodatevariations in interocular distance from viewer to viewer. In theseimplementations, a base interocular distance can be input into system100 d, for example at computing device 104 d, prior to a given viewerusing system 100 d.

In other implementations, as depicted in FIG. 17, processor 1103 d canbe enabled to insert two dimensional images 1701 into a stream 1703 ofstereoscopic images 105 d based on virtual distance 1503. Indeed, theinsertion of two dimensional images 1701 effectively means that theinterocular distance has been reduced to 0 mm. In some implementations,processor 1103 d can be enabled to insert two dimensional images 1701into stream 1703 of stereoscopic images 105 d when virtual distance 1503is one or more of above a first given threshold value and below a secondgiven threshold value. For example, above a given threshold value ofvirtual distance 1503, such as 1000 meters, the very little parallaxwill be present in stereoscopic images 105 c, and hence switching to atwo dimensional view via insertion of two dimensional images 1701 intostream 1703 of stereoscopic images 105 d can reduce processing time. Itis appreciated that any suitable number of two dimensional images 1701can be inserted into stream 1703 and for any suitable length of timedepending, for example, on virtual distance 1703.

Similarly, below a second given threshold value there can be situationswhere a three dimensional image may not be suitable or desired, andhence, system 100 d can switch to a two dimensional mode. Switching to atwo dimensional mode can save processing time, for example for higherframe rates and/or when processing many features in a scene (e.g.processing many polygons).

In some implementation processor 1103 d is enabled to insert twodimensional images 1701 into stream 1703 of stereoscopic images 105 c byreplacing one of the left eye images 106 ad and the right eye images 106bd with the other of right eye images 106 bd and the left eye images 106ad. In other words, each of the left eye and right eye are provided withthe same image.

Indeed, it is now apparent that varying of interocular distance,including but not limited to setting interocular distance to 0 mm for atwo dimensional mode, can comprise an interactive real time control ofinterocular distance to enable various features, including but notlimited to:

1. Quickly switching and recalibrating from one viewer to another viewerwith different interocular distance, so that the life time scalingeffects and distance judgements are relevant to a given viewers trainingat his/her particular interocular distance throughout his/her lifetime

2. Quickly switching from a three dimensional mode at a “normal”interocular distance to a two dimensional mode to save bandwidth orwhere only very long distances are visible.

3. Changing the “feel” and/or view of system 100 d by switching betweena normal mode (interocular distance at a baseline) to a hyperstereo mode(interocular distance greater than a baseline) or a hypostereo mode(interocular distance less than a baseline).

While various implementations are described herein in isolation, it isappreciated, for example, that head tracking and interocular distancevariation can be combined in any suitable combination as desired.

Collimated stereo display systems have been described herein to providerelief for eye strain when viewing three dimensional images, withvarious further improvements for improving the three dimensional viewingexperience.

Those skilled in the art will appreciate that in some implementations,the functionality of system 100, 100 a, 100 b, 100 c, 100 d can beimplemented using pre-programmed hardware or firmware elements (e.g.,application specific integrated circuits (ASICs), electrically erasableprogrammable read-only memories (EEPROMs), etc.), or other relatedcomponents. In other implementations, the functionality of 100, 100 a,100 b, 100 c, 100 d can be achieved using a computing apparatus that hasaccess to a code memory (not shown) which stores computer-readableprogram code for operation of the computing apparatus. Thecomputer-readable program code could be stored on a computer readablestorage medium which is fixed, tangible and readable directly by thesecomponents, (e.g., removable diskette, CD-ROM, ROM, fixed disk, USBdrive). Furthermore, it is appreciated that the computer-readableprogram can be stored as a computer program product comprising acomputer usable medium. Further, a persistent storage device cancomprise the computer readable program code. It is yet furtherappreciated that the computer-readable program code and/or computerusable medium can comprise a non-transitory computer-readable programcode and/or non-transitory computer usable medium. Alternatively, thecomputer-readable program code could be stored remotely buttransmittable to these components via a modem or other interface deviceconnected to a network (including, without limitation, the Internet)over a transmission medium. The transmission medium can be either anon-mobile medium (e.g., optical and/or digital and/or analogcommunications lines) or a mobile medium (e.g., microwave, infrared,free-space optical or other transmission schemes) or a combinationthereof.

Persons skilled in the art will appreciate that there are yet morealternative implementations and modifications possible, and that theabove examples are only illustrations of one or more implementations.The scope, therefore, is only to be limited by the claims appendedhereto.

What is claimed is:
 1. A system comprising: a processor; an imagegenerator configured to provide a stream of stereoscopic imagescomprising left eye images and right eye images; a projector comprisingat least one image modulator configured to receive the stream ofstereoscopic images from the image generator and form light into thestereoscopic images; a viewing apparatus configured to filter the lightinto the left eye images and the right eye images respectively therebyproviding a three dimensional rendering of the stereoscopic images whenviewed through the viewing apparatus; collimation apparatus configuredto receive and collimate the light from the at least one imagemodulator, such that rays of light are substantially parallel when thestereoscopic images are received at the viewing apparatus; and, a curvedsurface, the stereoscopic images projected onto the curved surface forviewing and the processor is configured to adjust the stereoscopicimages based on: a geometry of the curved surface and motion parallaxcues based on a position of a viewer, the collimation apparatuscomprising a collimating mirror configured to reflect the light from thecurved surface such that reflected light is collimated and bothaccommodation and vergence cues in the stereoscopic images are generallyin sync, the motion parallax cues taking the place of the vergence cueswhen head movement of the viewer is detected, so that objects in thestereoscopic images are pushed further away by the collimating mirrorand infinity in the stereoscopic images looks like a real infinity out awindow using the viewing apparatus, with eyes of the viewer relaxed bothin vergence and focus.
 2. The system of claim 1, further comprising:head tracking apparatus configured to acquire a position of the viewingapparatus; and the processor further configured to adjust thestereoscopic images for viewing based on one or more of the position andthe orientation of the viewing apparatus.
 3. The system of claim 1,further comprising a memory for storing data indicative of the geometry.4. The system of claim 1, wherein the processor is further configured toadjust the stereoscopic images based on the geometry of the curvedsurface by: determining images for display, the images comprising thestereoscopic images when they are to be provided on a flat screen;determining a distortion of the flat screen based on the data indicativeof the geometry of the curved surface and the position of the viewingapparatus; and adjusting the images based on the distortion such thatthe stereoscopic images will appear undistorted when projected onto thecurved surface.
 5. The system of claim 4, wherein adjusting thestereoscopic images is repeated each time a change in position of theviewing apparatus is determined.